September 22, 2023
Challenges when deploying Shiny apps in production:
Goal: give you inspiration and tips on what is possible with Shiny and ShinyProxy
Docker makes it easy to create, deploy, and run applications using containers. It allows to combine an application with everything it needs, such as libraries, dependencies, configuration, … into a container image.
The Dockerfile is a set of instructions telling Docker how to create the Docker image. For example, which R version you need, which R packages etc.
Extensible architecture: working on additional (cloud) container runtimes
Who is already using Docker? What do you like?
There is a learning curve, but in general we notice data scientists are able to quickly learn and use Docker in a good way.
In addition, because Docker makes it easier to reproduce results, it’s very useful in (data) science.
Many data science departments are adopting containers.
In order to guide the discussion, we will use a working example:
interactiveDisplay
Package for enabling powerful shiny web displays of Bioconductor objects
https://www.bioconductor.org/packages/release/bioc/html/interactiveDisplay.html
FROM rocker/r-ver:4.3.1
RUN apt-get update && \
apt-get install -y --no-install-recommends libz-dev liblzma-dev libbz2-dev \
libcurl4-openssl-dev libxt6
RUN R -q -e 'install.packages("BiocManager"); BiocManager::install(version = "3.17")'
RUN R -q -e 'BiocManager::install(c("GenomicRanges", "Gviz"), version = "3.17")'
RUN R -q -e 'BiocManager::install(c("ggbio", "interactiveDisplay"), version = "3.17")'
COPY Rprofile.site /usr/local/lib/R/etc/
EXPOSE 3838
CMD ["R", "-q", "-e", "library(interactiveDisplay);data(mmgr);display(mmgr);"]
Often trial-and-error: both for CRAN and Bioconductor packages.
Fast, easy and good?
Full integration with apt as every binary resolves all its dependencies: No more installations (of pre-built archives) only to discover that a shared library is missing. No more surprises.
https://eddelbuettel.github.io/r2u/
Very promising solution.
Supports Bioconductor
FROM rocker/r2u:jammy
RUN R -q -e 'install.packages("BiocManager"); BiocManager::install(version = "3.17")'
RUN R -q -e 'BiocManager::install(c("GenomicRanges", "Gviz"), version = "3.17")'
RUN R -q -e 'BiocManager::install(c("ggbio", "interactiveDisplay"), version = "3.17")'
COPY Rprofile.site /usr/lib/R/etc/
EXPOSE 3838
CMD ["R", "-q", "-e", "library(interactiveDisplay);data(mmgr);display(mmgr);"]
Easy: ✔
Fast: ✔ 1m36s vs ️ 34m27s
Good: ✔
pak installs R packages from CRAN, Bioconductor, GitHub, URLs, git repositories, local files and directories. It is an alternative to install.packages() and devtools::install_github(). pak is fast, safe and convenient.
FROM rocker/r-ver:4.3.1
RUN apt-get update && \
apt-get install -y --no-install-recommends liblzma-dev libbz2-dev libxt6
RUN R -q -e 'install.packages("pak")'
RUN R -q -e 'pak::pkg_install(c("GenomicRanges", "Gviz", "ggbio", "interactiveDisplay"))'
COPY Rprofile.site /usr/local/lib/R/etc/
EXPOSE 3838
CMD ["R", "-q", "-e", "library(interactiveDisplay);data(mmgr);display(mmgr);"]
The renv package helps you create reproducible environments for your R projects. Use renv to make your R projects more isolated, portable and reproducible.
Reproducible: renv records the exact package versions you depend on, and ensures those exact versions are the ones that get installed wherever you go.
https://rstudio.github.io/renv/
FROM rocker/r-ver:4.3.1
RUN apt-get update && \
apt-get install -y --no-install-recommends libz-dev libbz2-dev liblzma-dev \
libcurl4-openssl-dev libxt6
RUN R -q -e 'install.packages("renv")'
RUN R -q -e 'renv:::renv_pak_init(stream = "devel", force = TRUE)'
RUN mkdir /app
COPY renv.lock /app
WORKDIR /app
RUN R -q -e 'renv::restore()'
COPY Rprofile.site /usr/lib/R/etc/
EXPOSE 3838
CMD ["R", "-q", "-e", "library(interactiveDisplay);data(mmgr);display(mmgr);"]
What tool do you use?
Which tool have you already tested?
docker build -t myregistry.com/interactive-display docker push myregistry.com/interactive-display
Works on Linux, MacOS and Windows.
Building and pushing can and should be fully automated:
Dockerfile in Gitgit commit && git push
Edit the application.yml file and restart ShinyProxy.
As usual, best practice to automate this:
application.yml in Gitgit commit && git pushspecs:
- id: interactiveDisplay
display-name: InteractiveDisplay
description: package for enabling powerful shiny web displays of Bioconductor objects
container-image: myregistry.com/interactive-display
In essence this is solved by using Docker images
Let’s deploy another app using an older R version:
FROM openanalytics/r-ver:3.6.3
RUN apt-get update && apt-get install -y --no-install-recommends \
libz-dev
RUN R -q -e "install.packages('shiny')"
CMD ["R", "-q", "-e", "shiny::runExample('01_hello')"]
Dockerfilespecs:
- id: reproducibility
max-instances: -1
container-image: |
"myregistry.com/img:#{proxy.getRuntimeObject('SHINYPROXY_PARAMETERS').getValue('version')}"
parameters:
definitions:
- id: version
default-value: v1.0.1
value-sets:
- values:
version:
- v1.0.1
- v1.0.0
- v0.9.0
How do you guarantee reproducibility when using Shiny apps?
Do you write tests for your Shiny app?
Many Shiny apps require a directory to store files.
ShinyProxy allows accessing a shared directory using volumes.
shared between multiple users:
specs:
- id: interactiveDisplay
container-image: myregistry.com/interactive-display
container-volumes: ["/mnt/interactive-display:/mnt/interactive-display"]unique to every user:
specs:
- id: interactiveDisplay
container-image: myregistry.com/interactive-display
container-volumes: ["/mnt/interactive-display/#{proxy.userId}:/mnt/interactive-display"]essential when using ShinyProxy for hosting IDEs (Rstudio, Jupyter notebook …)
Do you face any other challenges with persistence storage?
Shiny app can directly access any database service, using regular R code
because Docker isolates the network, accessing databases on the same server as ShinyProxy can be tricky, but is possible
best practice to pass database host, username and password as environment variables:
specs:
- id: interactiveDisplay
container-image: myregistry.com/interactive-display
container-env:
DB_HOST: "172.16.10.10"
DB_USERNAME: "postgres"
DB_PASSWORD: "myPassword!"Kubernetes secrets natively supported
Docker swarm secrets natively supported
⚠️ do not store the password in the R code or on git
Bonus: run pgadmin on ShinyProxy:
https://github.com/openanalytics/shinyproxy-pgadmin-demo
Proving a user’s identity
library(shiny)
shinyServer(function(input, output, session) {
output$username <- renderText({ Sys.getenv("SHINYPROXY_USERNAME") })
output$groups <- renderText({ Sys.getenv("SHINYPROXY_USERGROUPS") })
})
Giving a user permission to access or do something
ShinyProxy: whether a user can access an app:
specs:
- id: auth-demo
display-name: Demo of authentication and authorisation
container-image: myregistry.com/auth-demo
access-users:
- jack
access-groups:
- mathematicians
Shiny: whether a user can see some data or preform a certain action:
https://shinyproxy.io/documentation/usage-statistics/
https://github.com/openanalytics/shinyproxy-monitoring
Usage statistics, logging and audit logging supported.
Solution: container pre-initialization